Method and apparatus for summarization of dialogs

ABSTRACT

A method for summarizing dialogs may include obtaining an agent text stream and a customer text stream, segmenting the agent text stream and customer text stream into sentences, and labeling sentences associated with the segmented agent text stream and the segmented customer text stream. The method may further include extracting sentences from the agent text stream and the customer text stream based upon frequencies of appearance of words and terms of interest; generating an agent summary paragraph based on the extracted sentences from the agent text stream, and generating a customer summary paragraph based on the extracted sentences from the customer text stream. The method may identify keywords associated with each of the agent summary paragraph and the customer summary paragraph.

BACKGROUND INFORMATION

Businesses and other types of organizations often operate customerservice centers for handling customers' requests for descriptions ofand/or issue resolution regarding products, services, features andofferings, equipment setup and installation, performance issues,technical support, and/or other information (e.g., billing, etc.).Interactions with a customer can involve a customer servicerepresentative (CSR) exchanging a dialog with the customer to provideassistance with new or existing services. An audio recording of thedialog may be made, and translated to text to produce transcripts forsubsequent use. However, transcripts of such dialogs can be lengthy anddifficult to read, particularly if multiple CSRs are involved inassisting the customer over time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary environment associatedwith a customer service center using dialog summarization according toan embodiment;

FIG. 2 is a diagram illustrating an exemplary networking environmentincluding a device performing dialog summarization according to anembodiment;

FIG. 3 is a block diagram illustrating exemplary components of a deviceaccording to an embodiment;

FIG. 4 is a flowchart showing a process for providing summarization ofdialogs according to an embodiment;

FIG. 5 is a drawing showing an exemplary agent summary paragraph havingkeywords annotated with highlighting; and

FIGS. 6A-E are drawings illustrating examples of various interfaces forinteracting with summary paragraphs according to an embodiment.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings identify the same orsimilar elements. The following detailed description does not limit theinvention, which is defined by the claims.

For purposes of description, an end user may be referred to herein as a“customer” which is not intended to limit embodiments according to thisdisclosure to an end user being bound within a traditional definition ofa “customer.” Accordingly, the term “user device” may be referred tointerchangeably with the term “customer device.” Description of a livecustomer service representative (CSR), also referred to herein as a“service agent” or simply “agent,” may presume a communications devicethat can include a display proximal to the service agent whichtypically, but not necessarily, may be associated with a customerservice center (CSC). As used herein, the term “call” may be generalizedto any type of interactive communication between the customer and theservice agent, that may include, for example, voice, video, and/or textcommunications. Additionally or alternatively, embodiments may includeother non-interactive communications between the customer and theservice agent.

CSCs may receive many thousands of calls from customers seekingresolution of issues which may relate to products, services, featuresand offerings, promotions, equipment setup and installation, performanceissues, technical support, billing, etc. During each call an agent triesto identify the cause of customer's complaint and to resolve theproblem. Agents may write down important points on a piece of paper asthe call proceeds. At the end of the call, agents may consult the papernotes and summarize their customer conversation on a desktop applicationexecuting on a computer. In some instances, various details of theconversation get omitted or forgotten, particularly when a conversationflow keeps changing and when the conversation lasts for a long time. Onepartial remedy to this address problem is to leverage the audiorecording of each conversation and transcripts generated from thoserecordings. However, for most of the calls neither the audio recordingsnor the transcripts are being reviewed by the agents. Both therecordings and transcripts can be very lengthy, and the transcripts haveno punctuation, hence to the reader, each transcript looks like a streamof words and thus is difficult to parse. Moreover, the transcripts mayhave a word error rate that often can reach 40% and higher. Extractinginformation relevant for each conversation may be challenging given thescope of issues addressed by agents. Relevant information to extract mayinclude, for example, technical problems discussed, promises made to thecustomer, customer sentiment during the call, etc.

Accordingly, embodiments herein are directed to automatically generatingsummaries of dialogs between agents and customers to facilitatetechnical support services. The summaries may identify key words and keyphrases to ease the identification of topics and what promises wereextended to the customer during the call. In some embodiments, agraphical interface may highlight the identified key words and phrasesin various ways. The agent may interact with the summary to obtain morecontext from the dialogs and determine when various issues werediscussed during the customer support call. The generated summariesobviate the need for a CSR to read a long and intractable stream ofwords. Having the relevant information not only leads to higher customersatisfaction and helps to retain valuable customers, it also reduces thecall holding time if subsequent calls are needed, as the next (likely adifferent) agent may be able quickly review the summary of what had beenpreviously discussed without the customer having to reiterate thedescription of the technical issue.

FIG. 1 is a diagram illustrating an exemplary environment associatedwith a customer service center using dialog summarization according toan embodiment. Environment 100 may include a user device 110, a device120 associated with a customer service center (CSC) (hereinafter “CSCdevice 120”), a transcript generator 130 associated with a transcriptservice, and a text/language processor 140 with a storage device 150associated with a natural language processing (NLP) service. A customermay use user device 110 to contact CSC device 120 (e.g., communicationsrepresented by arrow 115). Over the course of the help session, theaudio representing the dialog between the CSR (i.e., agent) and thecustomer may be recorded by CSC device 120. CSC device 120 may forwardthe dialog data 125 to transcript generator 130. From dialog data 125,transcript generator 130 may create text data to generate transcripts135 that represent dialog data 125 in the form of “text streams” of boththe agent and the customer. Specifically, the text of dialog data 125 intranscripts 135 associated with the customer may be referred to hereinas an “customer text stream.” The text of dialog data 125 in transcripts135 associated with the agent may referred to herein as an “agent textstream.” Transcripts 135 may be stored in a file using any format, suchas, for example, ASCII text, binary, compressed, and/or encrypted, etc.

Further to FIG. 1, transcript generator 130 may forward the transcripts135 to text/language processor 140. Text/language processor 140 mayoperate on received transcripts 135 to perform various natural languagetasks, such as, for example, segmentation (e.g., applying punctuation),topic modeling, promise determination, sentiment analysis andsummarization of dialog. In an embodiment, text/language processor 140may generate summary paragraphs 145 that, may be annotated withkeywords, and split into multiple paragraphs corresponding to an agentsummary paragraph and/or a customer summary paragraph. The summaryparagraphs may be produced, for example, using an extractivesummarization approach which can be based on determining terms ofinterest belonging to distinct subject classes, and assigning frequencyrelated weights to emphasize the terms of interest in receivedtranscripts 135. The frequency related weights associated with the termsof interest may subsequently be used to determine weights of sentencesin received transcripts 135. The summary paragraph may be generatedusing sentences extracted from received transcripts 135 based on theaforementioned weights of sentences.

Summary paragraphs 145 may be provided back to CSC device 120 for use byCSR in assisting customers and/or for saved to create of record of thecustomer service sessions. Text/language processor 140 may include anNLP engine for performing various natural language tasks. For example,topic modeling, promises determination, and/or sentiment analysis can bewell-suited for operations by an NLP engine. To avoid high computationalrequirements of NLP, an embodiment may rely on outsourced naturallanguage understanding (NLU) for customer inputs to vendor-based NLPengines.

FIG. 2 illustrates an example networking environment in which systemsand/or methods for summarizing dialogs may be embodied. As shown in FIG.2, environment 200 may include user devices 110-1 through 110-N, CSCdevice 120-1 through 120-M, transcript generator 130, text/languageprocessor 140, NLP engine 220, summary interaction tool 240-1 through240-M, back-end network 250, and access network 260.

Text/language processor 140 may reside in back-end network 250, and befunctionally coupled to CSC devices 120, transcript generator 130, andNLP engine 220. In an embodiment, transcript generator 130 and NLPengine 220 may both reside in back-end network 250 as shown in FIG. 2.In an alternative embodiment, transcript generator 130 and/or NLP engine220 may reside in another network external to back-end network 250 (notshown), and be associated with one or more external service providers.In this alternative embodiment, transcript generator 130 and/or NLPengine 220 may communicate with backend network 250 through accessnetwork 260 or another network not shown in FIG. 2. Further referring toFIG. 2, transcript generator 130 may be functionally coupled to CSCdevices 120 within back-end network 250. Back-end network 250 mayexchange data with access network 260 to communicate with user devices110. User devices 110 may be wireless devices (e.g., mobile devices) andmay exchange data wirelessly with access network 260.

Further referring to FIG. 2, transcript generator 130 may receive fromCSC devices 120 digital audio recordings of dialogs between agents andcustomers. Each dialog may be generated from the data exchanged, viaaccess network 260, between user device 110 and CSC device 120 during aconversation occurring for a tech support call. The dialog data, whichincludes the digitized voices of the agent and the customer, may beprovided to transcript generator 130 to generate text data of thedialog. The text data may include the combined text streams of both theagent and the customer. The text data may be provided to text/languageprocessor 140, which may separate the text data from an agent textstream and a customer text stream. Text/language processor 140, inconjunction with NLP engine 220, may further segment the agent textstream and customer text stream into sentences, and perform furtherprocessing to reduce each segmented text stream into summary paragraphs.Text/language processor 140 may then identify keywords in each summaryparagraph and provide the paragraphs to CSC devices 120. Through CSCdevices 120, agents may interact with the summary paragraphs using thesummary interaction tool 240 based on the identified keywords.

User device 110 may include a device capable of communicating via anetwork, such as access network 260. For example, user device 110 maycorrespond to a mobile communication device (e.g., a smartphone, or apersonal digital assistant (PDA)), a portable computer device (e.g., alaptop computer, a tablet computer, a wearable computer), and/or anothertype of device. In some implementations, user device 110 may include aclient-side application to interact with agents during, for example, a“help desk” session through CSC device 120 via text, voice, video,and/or image data.

CSC device 120 may include a device capable of communicating via anetwork, such as back-end network 250 and/or access network 260. Forexample, CSC device 120 may include one or more computing devices, suchas a laptop computer, a desktop computer, a tablet computer, etc. CSCdevice 120 may be used by an individual within an organization, such asan agent within a CSC. In an embodiment, CSC device 120 may provide anagent with access to summary interaction tool 240 which may allow anagent to interact with summary paragraphs. For example, summaryinteraction tool 240 can provide a graphical user interface allowing anagent to interact with agent summary paragraphs and/or customer summaryparagraphs. Summary interaction tool 240 may accept standard mousecommands for controlling the cursor and selection objects and text(e.g., clicks, double-clicks, drags, selections, etc.). Summaryinteraction tool 240 may also accept keyboard commands issued by theagent, which may include, for example, standard key-strokes and/orshortcuts issued using command, shift, and/or control key combinations.Exemplary interfaces which may be provided to an on a display of CSCdevice 120 are described below in reference to FIGS. 5 though 6E.

Transcript generator 130 may include any device capable of exchangingdata via a network, such a, for example, back-end network 250. Forexample, transcript generator 130 may be implemented in one or morecomputing devices, and can include a personal computer, a server,mainframe, etc. In an embodiment, transcript generator 130 may be underthe control of a third party service provider, and transcripts for useby text/language processor 140 may be generated as a service.

Text/language processor 140 may include any device capable of exchangingdata via a network, such as, for example, back-end network 250.Text/language processor 140 may be implemented on one or more computingdevices, such as a desktop computer, server, a mainframe, etc.Text/language processor 140 may concurrently receive a plurality oftranscripts from transcript generator 130, perform various textmodelling task, such as, for example, topic modeling, promisedetermination, sentiment analysis, and/or summarization. Transcriptgenerator 130 may provide various results, including summary paragraphs,to multiple CSC devices 120 in a concurrent manner.

NLP engine 220 may include any device capable of communicating via anetwork, such as, for example, back-end network 250. NLP engine 220 beimplemented on one or more computing devices, such as a desktopcomputer, server, a mainframe, etc. NLP engine 220 may be an artificialintelligence (AI) based third-party vendor service, for example, IBMWatson, Amazon Lex™, or Google Dialogflow™, capable of processing datafrom a dialog session between a user of user device 110 and an agent ofCSC device 120. NLP engine 220 may analyze the dialog data and detecttopics, promises, and/or sentiments associated with the customer dialogsession. NLP engine 220 may provide the analysis of the dialog sessionto text/language processor 140.

Back-end network 250 may be a secure network providing a privateconnection space between network devices to communicate/share networktraffic. Back-end network 250 may include a local area network (LAN)and/or a wide area network (WAN), an intranet, or any combination ofnetworks. Some or all of back-end network 250 may be managed by aprovider of communication services that also manages access network 260and CSC devices 120. Back-end network 250 may allow the delivery of IPand/or non-IP services between network devices residing therein, andalso to user devices 110 via access network 260. Back-end network mayinterface with other external networks, such as, for example, theinternet. Back-end network 250 may include an IP Multimedia Sub-system(IMS) network (not shown in FIG. 2). An IMS network may include anetwork for delivering IP multimedia services and may provide mediaflows between user devices 110 and external IP networks or externalcircuit-switched networks (not shown in FIG. 2).

Access network 260 may include a wireless radio access network forexchanging data with user devices 110. For example, access network 260may include a fourth generation (4G) wireless network which may, forexample, utilize the Long Term Evolution (LTE) standard and thus includean evolved packet core (EPC) network. Additionally or alternatively,access network 260 may include an LTE Advanced (LTE-A) access networkand/or a fifth generation (5G) access network or other advanced networkthat includes functionality such as 5G new radio (NR) base stations;carrier aggregation; advanced or massive multiple-input andmultiple-output (MIMO) configurations (e.g., an 8×8 antennaconfiguration, a 16×16 antenna configuration, a 256×256 antennaconfiguration, etc.); cooperative MIMO (CO-MIMO); relay stations;Heterogeneous Networks (HetNets) of overlapping small cells andmacrocells; Self-Organizing Network (SON) functionality; machine typecommunication (MTC) functionality, such as 1.4 MHz wide enhanced MTC(eMTC) channels (also referred to as category Cat-M1), Low Power WideArea (LPWA) technology such as Narrow Band (NB) IoT (NB-IoT) technology,and/or other types of MTC technology; and/or other types of LTE-A and/or5G functionality. In other implementations, access networks 260 mayinclude a Code Division Multiple Access (CDMA) access network. Forexample, the CDMA access network may include a CDMA enhanced High RatePacket Data (eHRPD) network (which may provide access to an LTEnetwork).

The quantity of devices and/or networks in environment 200 is notlimited to what is shown in FIG. 2. In practice, environment 200 mayinclude additional devices and/or networks, fewer devices and/ornetworks, different devices and/or networks, or differently arrangeddevices and/or networks than illustrated in FIG. 2. Also, in someimplementations, one or more of the devices of environment 200 mayperform one or more functions described as being performed by anotherone or more of the devices of environment 200. Devices of environment200 may interconnect via wired connections, wireless connections, or acombination of wired and wireless connections.

FIG. 3 is a diagram illustrating example components of a device 300according to an embodiment. User device 110, CSC device 120,text/language processor 140, and/or NLP engine 220 may each include oneor more devices 300. As shown in FIG. 3, device 300 may include a bus310, a processor 320, a memory 330, an input device 340, an outputdevice 350, and a communication interface 360.

Bus 310 may include a path that permits communication among thecomponents of device 300. Processor 320 may include any type ofsingle-core processor, multi-core processor, microprocessor, latch-basedprocessor, and/or processing logic (or families of processors,microprocessors, and/or processing logics) that executes instructions.In other embodiments, processor 320 may include an application-specificintegrated circuit (ASIC), a field-programmable gate array (FPGA),and/or another type of integrated circuit or processing logic.

In some embodiments, additional processors (not shown) such as, forexample, artificial intelligence (AI) accelerators, neural netaccelerators, etc., may be included to facilitate the operation ofvarious AI applications and/or processes, such as, for example, neuralnet training and/or operation, natural language processing functionalityincluding, for example, topic modeling, promise determination (e.g.,determining if promises were made by the agent to the customer),sentiment analysis, summarization, etc.

Memory 330 may include any type of dynamic storage device that may storeinformation and/or instructions, for execution by processor 320, and/orany type of non-volatile storage device that may store information foruse by processor 320. For example, memory 330 may include a randomaccess memory (RAM) or another type of dynamic storage device, aread-only memory (ROM) device or another type of static storage device,a content addressable memory (CAM), a magnetic and/or optical recordingmemory device and its corresponding drive (e.g., a hard disk drive, asolid state drive, optical drive, etc.), and/or a removable form ofmemory, such as a flash memory.

Input device 340 may allow an operator to input information into device300. Input device 340 may include, for example, a keyboard, a mouse, apen, a microphone, a remote control, an audio capture device, an imageand/or video capture device, a touch-screen display, and/or another typeof input device. In some embodiments, device 300 may be managed remotelyand may not include input device 340. In other words, device 300 may be“headless” and may not include a keyboard, for example.

Output device 350 may output information to an operator of device 300.Output device 350 may include a display, a printer, a speaker, and/oranother type of output device. For example, output device 350 mayinclude a display, which may include a liquid-crystal display (LCD) fordisplaying content to the customer. In some embodiments, device 300 maybe managed remotely and may not include output device 350. In otherwords, device 300 may be “headless” and may not include a display, forexample.

Communication interface 360 may include a transceiver that enablesdevice 300 to communicate with other devices and/or systems via wirelesscommunications (e.g., radio frequency, infrared, and/or visual optics,etc.), wired communications (e.g., conductive wire, twisted pair cable,coaxial cable, transmission line, fiber optic cable, and/or waveguide,etc.), or a combination of wireless and wired communications.Communication interface 360 may include a transmitter that convertsbaseband signals to radio frequency (RF) signals and/or a receiver thatconverts RF signals to baseband signals. Communication interface 360 maybe coupled to one or more antennas/antenna arrays for transmitting andreceiving RF signals.

Communication interface 360 may include a logical component thatincludes input and/or output ports, input and/or output systems, and/orother input and output components that facilitate the transmission ofdata to other devices. For example, communication interface 360 mayinclude a network interface card (e.g., Ethernet card) for wiredcommunications and/or a wireless network interface (e.g., a WiFi) cardfor wireless communications. Communication interface 360 may alsoinclude a universal serial bus (USB) port for communications over acable, a Bluetooth™ wireless interface, a radio-frequency identification(RFID) interface, a near-field communications (NFC) wireless interface,and/or any other type of interface that converts data from one form toanother form.

Device 300 may perform operations directed to summarizing dialogsbetween the customers and the agents. Device 300, which may, in a singleunit or separately, correspond to CSC device 120 and/or text/languageprocessor 140. Device 300 may perform these operations in response toprocessor 320 executing software instructions contained in acomputer-readable medium, such as memory 330. For example, memory 330may store instructions corresponding to summary interaction tool 240,where upon execution by processor 320, device 300 (e.g., CSC 120)provides a user interface to an agent for interacting with summaryparagraphs, as will be described below in reference to FIG. 4. Inanother example, either within the same or a separate and distinctdevice 300, instructions in memory may cause device 300 (e.g.,text/language processor 140) to process agent text stream and/orcustomer text stream to: apply punctuation to generate sentences;perform indexing on each sentence, generate separate summariescorresponding to agent and the customer text streams, and identify keywords in the summary paragraphs.

In an embodiment, a computer-readable medium may be defined as anon-transitory memory device. A memory device may be implemented withina single physical memory device or spread across multiple physicalmemory devices. The software instructions may be read into memory 330from another computer-readable medium or from another device. Thesoftware instructions contained in memory 330 may cause processor 320 toperform processes described herein. Alternatively, hardwired circuitrymay be used in place of, or in combination with, software instructionsto implement processes described herein. Thus, implementations describedherein are not limited to any specific combination of hardware circuitryand software.

Although FIG. 3 shows exemplary components of device 300, in otherimplementations, device 300 may include fewer components, differentcomponents, additional components, or differently arranged componentsthan depicted in FIG. 3. Additionally, or alternatively, one or morecomponents of device 300 may perform one or more tasks described asbeing performed by one or more other components of device 300.

FIG. 4 is a flow chart illustrating an exemplary process 400 consistentwith an embodiment. Process 400 may be performed by device 300, whereone or more processor(s) 320 may execute instructions stored inmemory/memories 330 for summarizing dialogs and permitting agentinteraction with summary paragraphs. Process 400 may be performed by oneor more devices 300, which can be associated with CSC device 120,text/language processor 140, and/or NLP engine 220.

Processor 320 may obtain an agent text stream and a customer text stream(Block 410). In an embodiment, obtaining the agent text stream and/orthe customer text stream may include processor 320 receiving atranscript of a dialog between the agent and the customer, identifying afirst text stream within the transcript associated with the agent, andidentifying a second text stream within the transcript associated withthe customer. The transcript may be derived from an audio recording ofthe conversation between the agent and the customer. In an embodiment,the transcript of the dialog may separate the words spoken by each partyinto alternating channels (e.g., an agent channel and a customerchannel). The received transcript may be a very long stream of wordsthat can be difficult to read. The transcript may be generated bytranscript generator 130, which may be operated and/or under the controlof a third party service/contractor.

In an embodiment, processor 320 may identify a first text stream withinthe transcript associated with the agent (referred to herein as “agenttext stream” which corresponds to an “agent channel”), and identify asecond text stream within the transcript associated with the customer(referred to herein as “customer text stream” which corresponds to a“customer channel”). If other parties are present (additional customers,agents, etc.), these parties may be assigned to their own particulartext stream and channels. In practice, agent text stream and/or customertext stream may be a long and difficult to read string of words. Eachtext steam may go on for multiple pages and include high word errorrates. The identification of text streams may be performed with theassistance of NLP engine 220, and could be based on the recognition ofspecific words and/or phrases.

Processor 320 may segment the agent text stream and customer text streaminto sentences (Block 420). In an embodiment, segmenting the agent textstream and customer text stream into sentences may include assigningpunctuation to the first text stream and the second text stream using adeep neural network model, where the deep neural network model mayinclude, for example, a convolutional neural network model, abidirectional long term memory model with attention, and/or one or moretransformer models. Types of transformer models may include, forexample, a bidirectional encoder representations from transformers(BERT), a lite BERT (ALBERT), robustly optimized (RoBert), and/orembeddings from language models (ElMo). One type of transformer model,or a combination thereof, may be used in Block 420 for segmenting textstreams into sentences. In an embodiment, any combination of transformermodels may be used to segment text streams into sentences in Block 420.Accordingly, in Block 420, processor 320 produces a segmented agent textstream and a segmented customer text stream, each consisting ofsentences having punctuation.

Processor 320 may label sentences associated with the segmented agenttext stream and the segmented customer text stream (Block 430). In anembodiment, process 320 may label sentences by assigning a numericalindex to each sentence in the segmented agent text stream, and assigninga numerical index to each sentence in the segmented customer textstream. For example, a sentence with index N in the segmented agent textstream may be denoted by SA(N); a sentence with index N in the segmentedcustomer text stream may be denoted by SC(N). For example, in thesegmented agent text stream, SA(0) may be the first sentence, SA(1) maybe the second sentence, and so on. The segmented customer text streammay be similarly indexed.

Processor 320 may generate an agent summary paragraph based on thesegmented agent text stream, and a customer summary paragraph based onthe segmented customer text stream (Block 440). In an embodiment,generating summary paragraphs may include processor 320 extractingsentences from the agent text stream and/or the customer text streambased on frequencies of words in each text stream, and/or terms ofinterest in each text stream. Processor 320 may generate an agentsummary paragraph based on the extracted sentences from the agent textstream. Processor 320 may generate a customer summary paragraph based onthe extracted sentences from the customer text stream. The agent summaryparagraph and the customer summary paragraph produced by processor 320in Block 440 are reduced in size from the segmented agent text streamand the segmented customer text stream, respectively, and are thuseasier for an agent to read and understand. In an embodiment, a summaryparagraph may have a length of approximately 150-200 words. In anembodiment, processor 320 may apply a summarization model to thesegmented agent text stream and the segmented customer text stream,where the summarization model includes at least one of extractivesummarization based on a natural language tool kit (NLTK) package, or anabstractive summarization based on a transformer model.

For example, an extractive summarization can be generated, for example,by determining terms of interest belonging to distinct subject classes,and assigning frequency related weights to each word in a segmented textstream. The particular weights may be assigned to emphasize words thatare the terms of interest in the segmented text stream. In an example,subject matter classes may correspond to technical subjects, negativesentiments, promises, etc.

More specifically, in an embodiment, processor 320 may perform anextractive summarization by splitting words associated with vocabulariesinto a plurality of subsets, where each subset from the plurality ofsubsets may be associated with a distinct subject matter class. Anexemplary subset may include “special words” of interest associated witha customer service context (e.g., “WiFi,” “router,” “connection,” etc.).Other words frequently found in customer service dialogs, such as, forexample “hello,” “day,” “home,”, etc. may not be included in theaforementioned exemplary subset.

Processor 320 may assign weights to words corresponding to a frequencyof an appearance of the words in the segmented text stream, where theassigned weights include particular weights to emphasize wordsassociated with each subset. Weights may be represented by numericalvalues, and particular weighs may be given larger values to emphasizewords of interest. For example, a weight W_(f) may be assigned to a wordcorresponding to its frequency f of appearance in the text. Processor320 may assign weights in a manner where the aforementioned specialwords (e.g., “WiFi,” “router,” “connection,” etc.) are assigned largervalues (e.g., we add W=20 to its frequency based weight:W_(f)=W_(f)+W=W_(f)+20). In an embodiment, the value of W can befine-tuned to improve the accuracy of the summary paragraph.

Once the weights are specified, processor 320 may assign weights ofsentences based on the assigned weights to words. For example, eachsentence weight may be the average of the word weights determined forthe words from that sentence. For example, if the sentence is “HAVE ANICE DAY,” with word weights 1, 5, 4, and 2, then the sentence's weightmay be determined as (1+5+4+2)/4=3. Alternative calculation of sentenceweights can be done when by dropping (not using in the calculation) allstop words such as ‘A’, etc. In such a case, the sentence's weight wouldbe (1+4+2)/3=2.33.

Processor 320 may select a predetermined number of sentences based onthe assigned weights to words. For example, N (N being an integer)sentences having the highest weights may be selected, and then listed inchronological order to generate the summary paragraph. Processor 320 maythen list the selected sentences in an order to generate the summaryparagraph. For example, the order may correspond to a “chronological”order according to when the sentences appeared in the paragraph.Specifically, suppose that N=5 and the following sentences have thehighest weights: S(20), S(75), S(2), S(3), S(34). The summary paragraphmay be constructed as follows: S(2). S(3). S(20). S(34). S(75).

Referring again to FIG. 4, processor 320 may identify keywordsassociated with each of the agent summary paragraph and the customersummary paragraph (Block 450). In an embodiment, processor 320 may findkeywords relevant to topics of interest, where, for example, topics ofinterest may include technical information regarding equipment and/orservices associated with customer. Processor 320 may identify keywordsby matching words within summary paragraphs with sets of predefinedwords. The determination of the set predefined words may be driven bythe content of the help session. For example, if the CSC 120 is set upfor assisting customers with optical networking issues, sets ofpredefined keywords based on optical networking hardware and/orsoftware, information of interest, and/or prior experience with customerservice issues (e.g., keywords associated with problems commonlyencountered by customers). Once keywords have been matched, the keywordsmay be annotated in a display of the summary paragraphs for assistingthe agent in reading and/or scanning the agent summary paragraph and/orthe customer summary paragraph. In various embodiments, the annotationmay be accomplished using font size, underlining, font style, fontcolor, background color, highlighting, and/or animations (blinking,changing font size, font color, font type, adding graphics to the text,etc.) As used herein, “highlighting” annotates a word by overlaying theword with a surrounding color in a manner which allows the highlightedword to remain legible. As described in reference to FIG. 5, embodimentsmay annotate keywords with highlighting to quickly draw the attention ofthe agent to the keywords.

FIG. 5 shows an exemplary display of an agent summary paragraph 510having keywords annotated with highlighting. Additionally oralternatively, different forms of styling may be applied to the text.For example, variations in text style, color, size, line weight, fonttype, underlining, kerning, etc. may be applied. In some embodiments,font animation may be used (e.g., dynamic variations in text includingblinking, size changes, color changes, kerning changes, backgroundvariations (e.g., highlighting), and/or line thickness changes). In thecase of highlights overlaying text as displayed in FIG. 5, thehighlighting may be performed using different colors, patterns, and/orshades. In an embodiment shown in FIG. 5, highlighting may be displayedas gray area (e.g., a halftone pattern) overlaid on text. Accordingly,within agent summary paragraph 510, processor 320 found and highlightedkey words relevant to various information of interest. For example,processor 320 highlighted the keyword “box” because it is one of thepredefined keywords identifying a possible problem with a customer's settop box. As noted above, annotations can include combinations of variousfont size, underlining, font style, font color, background color, etc.Note that key words relevant to other type of information, such as, forexample promises made by the agent, can be highlighted using differentcombination of font, color, etc. The highlighting allows an agent who isreviewing the summary to easily identify key words of interest.

Referring back to FIG. 4, processor 320 may generate an interface forinteracting with the agent summary paragraph and/or the customer summaryparagraph based on the identified key words (Block 460). Processor 320may generate an interface for interacting with summary paragraphs bydetecting a selection of a key word in the agent summary paragraphand/or the customer summary paragraph. Processor 320 may identify anindex of a sentence in the segmented agent text stream and/or thesegmented customer text stream based on the detected selection of thekey word. Processor 320 may then display sentences corresponding to apredetermined range of indices from the segmented agent text streamand/or the segmented customer text stream, where the predetermined rangeof indices bound the identified index.

For example, in agent summary paragraph 510 shown in FIG. 5, if theagent hovers with a mouse cursor over a sentence with the keyword “box”in that sentence, processor 320 may identify the index of the sentencecorresponding to the keyword “box” in the associated segmented agenttext stream. If the index of the sentence associated with the keyword“box” is N, once the agent selects the keyword “box” in the sentence(e.g., performs a single mouse click on the keyword “box”), processor320 may display sentences SA(N−1), SA(N), and SA(N+1) from the segmentedagent text stream to provide more context around the sentence SA(N).

FIG. 6A shows an embodiment which displays a portion of text from asegmented text stream in a sub-window 620, where the portion of text isassociated with one or more sentences in summary paragraph 610. Whengenerating the interface to interact with at the agent summary paragraphand/or the customer summary paragraph, processor 320 may detect aselection of a sentence in the agent summary paragraph and/or thecustomer summary paragraph. Processor 320 may then identify an index ofthe detected sentence in the segmented agent text stream and/or thesegmented customer text stream based on the detected selection of thesentence. Processor 320 may then display neighboring sentencescorresponding to a predetermined range of indices from the segmentedagent text stream and/or the segmented customer text stream, where thepredetermined range of indices bound the identified index associatedwith the selected sentence. The neighboring sentences of the segmentedtext stream may be presented in a sub-window 620, which may graphicallyindicate an association with the detected sentence in the summaryparagraph (as shown, for example, using dotted lines in FIG. 6A). In anembodiment, an agent may double click while hovering over the detectedsentence in the summary paragraph, and processor 320 may retrievesentences S(N−2), S(N−1), S(N), S(N+1) S(N+2) from the segmented textstream for display in sub-window 620 to provide additional contextassociated with a selected sentence S(N) or keyword in sentence S(N).

FIG. 6B shows an embodiment which displays a portion of text from asegmented text stream in a sub-window 640 which is associated with textin summary paragraph 630, where the portion of text displayed isincreased if neighboring sentences in a segmented text stream are short.Accordingly, in an embodiment, processor 320 may determine a metriccorresponding to lengths of the neighboring sentences in a segmentedtext stream (e.g., segmented agent text stream and/or segmented customertext stream). Processor may increase a number of the neighboringsentences to display from the segmented text stream upon determining themetric corresponding to the length of the neighboring sentences is belowa threshold.

In summary, if the neighboring sentences in a segmented text stream areshort (i.e., less than X (e.g. a predetermined threshold, X=5) words inlength), the processor 320 may automatically add additional sentencesuntil their cumulative length exceeds desired number (T) of words. Forexample, as shown in FIG. 6B, an agent hovered over the sentence insummary paragraph 630 at a key phrase “you can just keep the cables” andclicked once. As shown in sub-window 640 in the segmented text stream,the closest sentence to the left, S(N−1), has only two words: “Youknow”, and therefore provides almost no additional information asdetermined by the processor 320. In that case, processor 320 mayretrieve the prior sentence (e.g., to the left in paragraph 630) andwill display sentences S(N−2), S(N−1), S(N), S(N+1).

A formula to determine the smallest index K of a sentence to bedisplayed from the segmented text stream may be expressed as follows:K=max{K:K<N & length(S(K)+S(K+1)+ . . . S(N−1))>T},

where:

-   -   N is an index of a sentence in the segmented text stream which        is the default number (e.g., N=1) of sentences to display to the        left and to the right of a target sentence,    -   T is a threshold specifying a minimum cumulative number of words        that are present in the displayed sentences,    -   length(S(k)) returns the length of the number of words in a        sentence S(K), and    -   “+” is an operation which means text concatenation.    -   In the example shown in FIG. 6B, T=10 and K=N−2.

Note that when the number of words is small (e.g., due to shortsentences to the left or right of a target sentence), more sentences maybe added to provide sufficient context. In an embodiment, K may exceed N(e.g., K=2 when N=1), but K should only be as large as needed to obtaina sufficient number of words for display (e.g., threshold T).

Moreover, if the sentences to the right are too short, processor 320 mayautomatically retrieve additional sentences S(N+1), S(N+2), . . . S(K)from a segmented text stream until their cumulative length exceeds Twords. In that case, the formula to determine index K is as follows:K=min{K:K>N & length(S(N+1)+S(N+2)+ . . . S(K))>T}.

One of ordinary skill in the art would appreciate the embodiments shownin FIGS. 6A and 6B are illustrative examples, and the actual window andsub-window locations can be changed and displayed in any manner. Forexample, FIG. 6C shows an alternative presentation with sub-window 660,having different positioning with respect to summary paragraph 650.Additionally, the phrase “Basically, you can just keep the cables.” ispresented differently in sub-window 640 and sub-window 660.

FIG. 6D shows an embodiment which displays a portion of text from asegmented text stream in a sub-window 680, where the portion of text isassociated with a sentence in summary paragraph 670. In an embodiment,processor 320 may detect a selection of a key word associated with afirst sentence in the agent summary paragraph and/or the customersummary paragraph, and identify a predetermined number of sentencesproximate to the first sentence in the segmented agent text streamand/or the segmented customer text stream, where the proximate sentencesinclude the selected key word. Processor 320 may display the firstsentence and the sentences proximate to the first sentence. Accordingly,another way of providing additional insight into the discussion of aparticular topic may be to show the nearest sentences with the matchingkey words. For example, if an agent wants to learn more about the of‘box’-discussion as indicated by the third sentence in the summary, theagent could press a key (for example the SHIFT key), hover the mouseover the word “box” in that sentence and click once. The system willdisplay two additional sentences containing the word ‘box’ from theagent spoken text before and after the target sentence

FIG. 6E shows an embodiment which displays a portion of text from asegmented text stream in a sub-window 690, where the portion of text isassociated with a sentence in summary paragraph 685. In the embodimentshown in FIG. 6E if the agent performs a combination of predefinedactions, such as a double click while pressing the shift key andhovering the mouse over the target word, then processor 320 may displayfour additional sentences with the target word (if possible) such as twosentences that came before the target sentence (or word) and twosentences that came after as provided in a segmented text stream. Thisembodiment may provide more context of the dialog between the agent andthe customer.

In other embodiments, the agent may use various commands (e.g., keycombinations and/or mouse commands) to access other forms of dataassociated with target work and/or sentence. For example, the agent mayelect to play the audio of the dialog correlated (associated) with akeyword and/or a sentence in a summary paragraph (instead of, or inaddition to, a segmented text stream. This may be useful in assistingthe agent in clarifying confusing text that may be associated withtranscription errors. In another embodiment, key words and/or sentencesmay be used as hyperlinks to access additional information associatedwith the summary paragraphs.

In the preceding, various preferred embodiments have been described withreference to the accompanying drawings. It will, however, be evidentthat various modifications and changes may be made thereto, andadditional embodiments may be implemented, without departing from thebroader scope of the invention as set forth in the claims that follow.The specification and drawings are accordingly to be regarded in anillustrative rather than restrictive sense.

For example, while a series of blocks have been described with respectto FIG. 4, the order of the blocks may be modified in otherimplementations. Further, non-dependent blocks may be performed inparallel.

It will be apparent that systems and/or methods, as described above, maybe implemented in many different forms of software, firmware, andhardware in the implementations illustrated in the figures. The actualsoftware code or specialized control hardware used to implement thesesystems and methods is not limiting of the embodiments. Thus, theoperation and behavior of the systems and methods were described withoutreference to the specific software code—it being understood thatsoftware and control hardware can be designed to implement the systemsand methods based on the description herein.

Further, certain portions, described above, may be implemented as acomponent that performs one or more functions. A component, as usedherein, may include hardware, such as a processor, an ASIC, or a FPGA,or a combination of hardware and software (e.g., a processor executingsoftware).

It should be emphasized that the terms “comprises”/“comprising” whenused in this specification are taken to specify the presence of statedfeatures, integers, steps or components but does not preclude thepresence or addition of one or more other features, integers, steps,components or groups thereof.

The term “logic,” as used herein, may refer to a combination of one ormore processors configured to execute instructions stored in one or morememory devices, may refer to hardwired circuitry, and/or may refer to acombination thereof. Furthermore, a logic may be included in a singledevice or may be distributed across multiple, and possibly remote,devices.

For the purposes of describing and defining the present invention, it isadditionally noted that the term “substantially” is utilized herein torepresent the inherent degree of uncertainty that may be attributed toany quantitative comparison, value, measurement, or otherrepresentation. The term “substantially” is also utilized herein torepresent the degree by which a quantitative representation may varyfrom a stated reference without resulting in a change in the basicfunction of the subject matter at issue.

To the extent the aforementioned embodiments collect, store, or employpersonal information of individuals, it should be understood that suchinformation shall be collected, stored, and used in accordance with allapplicable laws concerning protection of personal information.Additionally, the collection, storage and use of such information may besubject to consent of the individual to such activity, for example,through well known “opt-in” or “opt-out” processes as may be appropriatefor the situation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

No element, act, or instruction used in the present application shouldbe construed as critical or essential to the embodiments unlessexplicitly described as such. Also, as used herein, the article “a” isintended to include one or more items. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

What is claimed is:
 1. A method comprising: obtaining an agent textstream and a customer text stream; segmenting the agent text stream andcustomer text stream into sentences; labeling sentences associated withthe segmented agent text stream and the segmented customer text stream;extracting sentences from the agent text stream and the customer textstream based upon frequencies of appearance of words and terms ofinterest; generating an agent summary paragraph based on the extractedsentences from the agent text stream; generating a customer summaryparagraph based on the extracted sentences from the customer textstream; and identifying keywords associated with the agent summaryparagraph and the customer summary paragraph.
 2. The method of claim 1,wherein obtaining an agent text steam and customer text streamcomprises: receiving a transcript of a dialog between the agent and thecustomer; identifying a first text stream within the transcriptassociated with the agent; and identifying a second text stream withinthe transcript associated with the customer.
 3. The method of claim 2,wherein segmenting the agent text stream and customer text stream intosentences comprises: assigning punctuation to the first text stream andthe second text stream using a deep neural network model, wherein thedeep neural network model includes at least one of a convolutionalneural network model, a bidirectional long term memory model, or atransformer model.
 4. The method of claim 1, wherein labeling sentencesassociated with the agent text stream and the customer text streamcomprises: assigning an index to each sentence in the segmented agenttext stream and to each sentence in the segmented customer text stream.5. The method of claim 1, wherein generating the agent summary paragraphand the customer summary paragraph comprises: applying a summarizationmodel to the segmented agent text stream and the segmented customer textstream, wherein the summarization model includes at least one ofextractive summarization model based on a natural language tool kit, oran abstractive summarization model based on a transformer model.
 6. Themethod of claim 5, wherein applying the extractive summarization modelcomprises: determining the terms of interest belonging to distinctsubject classes; assigning frequency related weights to each word in asegmented text stream, wherein particular weights are assigned toemphasize words that are the terms of interest in the segmented textstream.
 7. The method of claim 6, further comprising: splitting wordsassociated with vocabularies into a plurality of subsets, wherein eachsubset from the plurality of subsets is associated with a distinctsubject matter class; assigning weights to words corresponding to afrequency of an appearance of the words in the segmented text stream,wherein the assigned weights include particular weights to emphasizewords associated with each subset; assigning weights to sentences basedon the assigned weights to words; selecting a predetermined number ofsentences based on the weights assigned to sentences; and listing theselected sentences in an order to generate the summary paragraph.
 8. Themethod of claim 1, wherein identifying keywords associated with each ofthe agent summary paragraph and the customer summary paragraph furthercomprise: finding keywords relevant to topics of interest, whereintopics of interest include technical information regarding equipment orservices associated with customer.
 9. The method of claim 1, furthercomprising: generating an interface for interacting with at least one ofthe agent summary paragraph or the customer summary paragraph based onthe identified key words, wherein the interacting comprises: detecting aselection of a key word in at least one of the agent summary paragraphor the customer summary paragraph; identifying an index of a sentence inat least one of the segmented agent text stream or the segmentedcustomer text stream based on the detected selection of the key word;and displaying sentences corresponding to a predetermined range ofindices from at least one of the segmented agent text stream or thesegmented customer text stream, wherein the predetermined range ofindices bound the identified index.
 10. The method of claim 9, whereinthe interacting with at least one of the agent summary paragraph or thecustomer summary paragraph based on the identified key words comprises:detecting a selection of a sentence in at least one of the agent summaryparagraph or the customer summary paragraph; identifying an index of thedetected sentence in at least one of the segmented agent text stream orthe segmented customer text stream based on the detected selection ofthe sentence; and displaying neighboring sentences corresponding to apredetermined range of indices from at least one of the segmented agenttext stream or the segmented customer text stream, wherein thepredetermined range of indices bound the identified index associatedwith the selected sentence.
 11. The method of claim 10, whereindisplaying neighboring sentences corresponding to a predetermined rangeof indices from at least one of the agent summary paragraph or thecustomer summary paragraph comprises: determining a metric correspondingto lengths of the neighboring sentences; and increasing a number of theneighboring sentences to display upon determining the metriccorresponding to the length of the neighboring sentences is below athreshold.
 12. The method of claim 9, wherein the interacting with atleast one of the agent summary paragraph or the customer summaryparagraph based on the identified key words comprises: detecting aselection of a key word associated with a first sentence in at least oneof the agent summary paragraph or the customer summary paragraph;identifying a predetermined number of sentences proximate to the firstsentence in at least one of the segmented agent text stream or thesegmented customer text stream, wherein the proximate sentences includethe selected key word; and displaying the first sentence and thepredetermined number of sentences proximate to the first sentence.
 13. Adevice comprising: a communication interface; a memory for storinginstructions; and a processor configured to execute the instructions to:obtain an agent text stream and a customer text stream; segment theagent text stream and customer text stream into sentences; labelsentences associated with the segmented agent text stream and thesegmented customer text stream; extract sentences from the agent textstream and the customer text stream based on frequencies of appearanceof words and terms of interest; generate an agent summary paragraphbased on the extracted sentences from the agent text stream; generate acustomer summary paragraph based on the extracted sentences from thecustomer text stream; and identify keywords associated with the agentsummary paragraph and the customer summary paragraph.
 14. The device ofclaim 13, wherein the instructions to obtain an agent text steam andcustomer text stream further cause the processor to: receive atranscript of a dialog between the agent and the customer; identify afirst text stream within the transcript associated with the agent; andidentify a second text stream within the transcript associated with thecustomer.
 15. The device of claim 14, wherein the instructions tosegment the agent text stream and customer text stream into sentencesfurther cause the processor to: assign punctuation to the first textstream and the second text stream using a deep neural network model,wherein the deep neural network model includes at least one of aconvolutional neural network model, a bidirectional long term memorymodel, or a transformer model.
 16. The device of claim 13, wherein theinstructions to label sentences associated with the agent text streamand the customer text stream further cause the processor to: assign anindex to each sentence in the segmented agent text stream and to eachsentence in the segmented customer text stream.
 17. The device of claim13, wherein the instructions to generate the agent summary paragraph andthe customer summary paragraph further cause the processor to: apply asummarization model to the segmented agent text stream and the segmentedcustomer text stream, wherein the summarization model includes at leastone of extractive summarization based on a natural language tool kit, oran abstractive summarization based on a transformer model.
 18. Thedevice of claim 14, further comprising instructions to: generate aninterface for interacting with at least one of the agent summaryparagraph or the customer summary paragraph based on the identified keywords, which further cause the processor to: detect a selection of a keyword in at least one of the agent summary paragraph or the customersummary paragraph; identify an index of a sentence in at least one ofthe segmented agent text stream or the segmented customer text streambased on the detected selection of the key word; and display sentencescorresponding to a predetermined range of indices from at least one ofthe segmented agent text stream or the segmented customer text stream,wherein the predetermined range of indices bound the identified index.19. The device of claim 18, wherein the instructions to generate aninterface for interacting with at least one of the agent summaryparagraph or the customer summary paragraph based on the identified keywords further cause the processor to: detect a selection of a sentencein at least one of the agent summary paragraph or the customer summaryparagraph; identify an index of the detected sentence in at least one ofthe segmented agent text stream or the segmented customer text streambased on the detected selection of the sentence; and display neighboringsentences corresponding to a predetermined range of indices from atleast one of the segmented agent text stream or the segmented customertext stream, wherein the predetermined range of indices bound theidentified index associated with the selected sentence.
 20. Anon-transitory computer-readable medium including instructions that,when executed by a processor of a device, cause the device to: obtain anagent text stream and a customer text stream; segment the agent textstream and customer text stream into sentences; label sentencesassociated with the segmented agent text stream and the segmentedcustomer text stream; extract sentences from the agent text stream andthe customer text stream based upon frequencies of appearance of wordsand terms of interest; generate an agent summary paragraph based on theextracted sentences from the agent text stream; generate a customersummary paragraph based on the extracted sentences from the customertext stream; and identify keywords associated with each of the agentsummary paragraph and the customer summary paragraph.